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Abstract. In this paper, we consider the problem of controller design using approximately bisimilar 
f*"*) abstractions with an emphasis on safety and reachability specifications. We propose abstraction-based 

O^l approaches to solve both classes of problems. We start by synthesizing a controller for an approxi- 

mately bisimilar abstraction. Then, using a concretization procedure, we obtain a controller for our 
initial system that is proved "correct by design". We provide guarantees of performance by giving 
K— 5 estimates of the distance of the synthesized controller to the maximal (i.e the most permissive) safety 

controller or to the time-optimal reachability controller. Finally, we use the presented techniques 
I combined with discrete approximately bisimilar abstractions of switched systems developed recently, 

for switching controller synthesis. 

1. Introduction 

O The use of discrete abstractions has become a standard approach to hybrid systems design [R098, 

IMR991 IHvSOll ITP061 IKB06L lRei09] . The benefit of this approach is double. Firstly, by abstracting 
the continuous dynamics, controller synthesis problems can be efficiently solved using techniques 
developed in the areas of supervisory control of discrete-event systems |RW87j or algorithmic game 
^ theory |AVW03j . Secondly, if the behaviors of the original system and of the discrete abstraction 

are formally related by an inclusion or equivalence relationship, the synthesized controller is known 
to be correct by design and thus the need of formal verification is reduced. Abstraction, using 
traditional systems behavioral relationships, relies on inclusion or equality of observed behaviors. 
One of the most common notions is that of bisimulation equivalence |Mil89| . However, for systems 
observed over metric spaces, requiring strict equality of observed behaviors is often too strong. 
Indeed, the class of continuous or hybrid systems admitting bisimilar discrete abstractions is quite 
restricted [AHLPOO, Tab09j. In |GP07] , a notion of approximate bisimulation, which only asks for 
closeness of observed behaviors, was introduced. This relaxation made it possible to extend the class 
of systems for which discrete abstractions can be computed [PGT081 IGPT10| . 

This paper deals with the synthesis of controllers using approximately bisimilar abstractions with an 
emphasis on safety and reachability problems. Safety problems consist in synthesizing a controller 
that restricts the behaviors of a system so that its outputs remain in some specified safe set. One is 
usually interested in designing a controller that is as permissive as possible since this makes it possible, 
using modular approaches, to ensure, a posteriori, secondary control objectives (see e.g. [RW87J). 
Reachability problems consist in synthesizing a controller that steers the observations of the system 
to some target region while keeping them in a given safe set along the way. In addition, in order to 
choose among the possible controllers, we try to minimize the time to reach the target. Hence, we 
consider a time-optimal control problem. We propose abstraction-based approaches to solve both 
classes of problems. We start by synthesizing a controller for an approximately bisimilar abstraction 



> 



This work was supported by the Agence Nationale de la Recherche (VEDECY project - ANR 2009 SEGI 015 01). 

1 



2 



ANTOINE GIRARD 



of our concrete system. Then, using a concretization procedure that is problem-specific, we obtain 
a controller for our concrete system that is proved "correct by design". For safety problems, we 
provide estimates of the distance between the synthesized controller and the maximal (i.e the most 
permissive) safety controller. For reachability problems, we provide estimates of the distance between 
the performances of the synthesized controller and of the time-optimal controller. As an illustration, 
we use these techniques in combination with the discrete approximately bisimilar abstractions of 
switched systems developed in |GPT10j . for switching controller synthesis. Preliminary versions 
of these results appeared in the conference papers [GirlOa] IGirlObj . The presentation has been 
improved and new results on estimates of the distance to maximal or optimal controllers have been 
added. Deeper numerical experiments have also been provided. 

Controller synthesis using approximately (bi) similar abstractions has also been considered in |TI08| 
IMT10| . In |TI08] . the authors use approximately bisimilar abstractions to design a suboptimal 
controller for a fixed bounded horizon optimal control problem. In this paper, we consider time- 
optimal control for reachability specifications, thus the time horizon is variable. Our work is more 
closely related to [MTlOj where time-optimal control is considered as well. We shall discuss further 
in the paper the main differences with our approach. Regarding safety specifications, this is the first 
paper proposing a specific approach using approximately bisimilar abstractions. 

2. Preliminaries 

We start by introducing the class of transition systems which serves as a common abstract modeling 
framework for discrete, continuous or hybrid systems (see e.g. [AHLPOOl ITab09] ). 

Definition 2.1. A transition system is a tuple T = (Q, L, 5, 0, H) consisting of a set of states Q; a 
set of actions L; a transition relation 5 Q Q x L x Q; a set of observations O; an output function 
H : Q —7- O. T is said to be discrete if Q and L are finite or countable sets, metric if the set of 
observations O is equipped with a metric d. 

The transition (q,l,q') G S will be denoted q' G 5(q,l); this means that the system can evolve from 
state q to state q' under the action I. Given a subset of actions V C L, we denote 5(q,L') = 
U/eL' ^(QjO- An action I G L belongs to the set of enabled actions at state q, denoted Enab(g), if 
S(q, I) 7^ 0. If Enab(g) = 0, then q is said to be a blocking state; otherwise it is said to be non- 
blocking. If all states are non-blocking, we say that the transition system T is non-blocking. The 
transition system is said to be non- deterministic if there exists q G Q and I G Enab((7) such that 
5(q, I) has several elements. A trajectory of the transition system is a finite sequence of states and 
actions (qo,lo), (qi,h), (qn-iJn-i),Qn where q i+ i G 8(qi,k) for alH G {0, . . . , JV - 1}. JV6N 
is referred to as the length of the trajectory. The observed behavior associated to trajectory is the 
finite sequence of observations oqo\02 ■ ■ ■ on where Oi = H(qi), for all i € {0, ... , A^}. 

This paper deals with controller synthesis for transition systems; we shall consider only static (i.e. 
without memory) state-feedback controllers. However, we will just use the term controller for brevity. 

Definition 2.2. A controller for transition system T is a map S : Q — > 2 L . It is well-defined if 
S(q) C Enab(g), for all q £ Q. The dynamics of the controlled system is described by the transition 
system T$ = (Q, L, 5s, O, H) where the transition relation is given by q' G 5s(q,l) if and only if 
/ G S(q) and q' G 5(q,l). 

Given a subset of states Q' C Q, we will denote S(Q') = \J gG Q, S(q). Let us remark that a state q 
of T$ is non-blocking if and only if S(q) 7^ 0. A controller essentially executes as follows. The state 
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q of T is measured, an action I £ S(q) is selected and actuated. Then, the system takes a transition 
q' G S(q, I), this is always possible if S is well-defined. 

In this paper, we consider approximate equivalence relationships for transition systems defined by 
approximate bisimulation relations introduced in [GP07J. 

Definition 2.3. Let T{ = (Qi,L, 5i, O, Hi), i = 1,2, be two metric transition systems with the same 
sets of actions L and observations O equipped with the metric d, let e > be a given precision. 
A relation R C Q\ x Q 2 is said to be an e -approximate bisimulation relation between Ti and T2 if, 
for all (91,92) G -R: 

• d(H 1 (q l ),H 2 (q 2 ))<e; 

• V7 G Enabi(9i), V91 G £1(91,/), 39 2 G 5 2 (q 2 ,l), sucn that (9i>92) e 

• V7 G Enab 2 (92), V9 2 G 5 2 (q 2 ,l), ^ G 61(91, Z), such that (91,92) G 

The transition systems Ti and T 2 are said to be approximately bisimilar with precision e, denoted 
Ti ~ e T 2 , if: 

• V91 G Qi, 392 G Q 2 , such that (91, 92) € R; 

• V92 G Q2, 39i G Qi, such that (91,92) G i?. 

If Ti is a system we want to control and T2 is a simpler system that we want to use for controller 
synthesis, then T 2 is called an approximately bisimilar abstraction of T\. 

We will denote for 91 G Qi, R(qi) = {92 G Q 2 \ (91, 92) G i?} and for Q[ C Q 1? i?(Qi) = U 9l6 Q' ^(^l); 
for 92 G Q 2 , R~ 1 (q2) = Ui G Qi\ (91,92) G fl} and for Q' 2 C Q 2 , i?" 1 ^) = U g2eQ ^ ^H^)' 

Remark 2.4. We assume that systems Ti and T2 have the same sets of actions and that matching 
transitions in the second and third items of the previous definition share the same input. These 
conditions can actually be relaxed using the notion of alternating approximate bisimulation rela- 
tion [Tab09j. The results presented in this paper can be easily extended to that setting. 

The problem of computing approximately bisimilar discrete abstractions has been considered for 
nonlinear control systems [PGT08] and switched systems jGPTlOj . A controller designed for an ab- 
straction can be used to synthesize a controller for the concrete system via a natural concretization 
procedure, described in jTab09llGirl0a] . which essentially renders the two controlled systems approx- 
imately bisimilar. This is the approach used in |PGT08| IGPT10"! IMTlOj . However, the controller 
for the concrete system obtained via this concretization procedure has several drawbacks. Firstly, it 
is generally a dynamic state-feedback controller (i.e. the controller has a memory) when it is known 
that for some control specifications such as safety [RW87J or reachability [BcrOO], it is sufficient to 
consider static state-feedback controllers. Secondly, the implementation of this controller requires 
the encoding of the dynamics of the abstraction which may result in a higher implementation cost. 
Thirdly, if the abstraction is deterministic, then there is essentially no more feedback: at each step 
the controller selects the control action and its internal state independently from the state of the 
concrete system. This may cause some robustness issues in case of unmodeled disturbances or fault 
occurrences. In the following, we present specific controller concretization procedure for safety and 
reachability specifications which do not suffer from the previous drawbacks. These techniques are 
readily applicable using the discrete abstractions mentioned above. 
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3. Controller Synthesis for Safety Specifications 

3.1. Problem Formulation. Let T = (Q, L, 5, O, H) be a transition system, let O s C O be a set of 
outputs associated with safe states. We consider the synthesis problem that consists in determining 
a controller that keeps the output of the system inside the specified safe set O s . 

Definition 3.1. A controller 5 for T is a safety controller for specification O s if, for all non-blocking 
states qo of the controlled system T$ (i.e. S(qo) ^ 0), for all trajectories of T$ starting from qo, 
(Wo), (qi,h), (?jv_i,/jv-i))?jv; for all i G {0,...,N}, H(qi) G O s and q N is a non-blocking 
state of T s (i.e. ^ 0). 

The condition that all trajectories end in a non-blocking state ensures that, starting from a non- 
blocking state, the controlled system can evolve indefinitely while keeping its output in the safe set 
O s . It is easy to verify by induction that an equivalent characterization of safety controllers is given 
as follows: 

Lemma 3.2. A controller S forT is a safety controller for specification O s if and only if for all non- 
blocking states q of the controlled system Ts (i.e. S{q) ^ 0j; H(q) G O s and for all q' G 5(q,S(q)), 
q' is a non-blocking state of Ts (i.e. S(q') / %). 

There are in general several controllers that solve the safety problem. We are usually interested in 
synthesizing a controller that enables as many actions as possible. This notion of permissivity can 
be formalized by defining a partial order on controllers. 

Definition 3.3. Let S\ and 52 be two controllers for transition system T, S\ is more permissive 
than 52, denoted 52 < S\, if for all q G Q, 52(g) C S\(q). The controller 5* for T is the maximal 
safety controller for specification O s , if 5* is a safety controller for specification O s , and for all safety 
controllers 5 for specification O s , 5 X 5*. 

It is well known that the maximal safety controller exists, is unique and can be computed using a 
fixed point algorithm (see e.g. jRW871 ITab 09j ) . This algorithm is guaranteed to terminate in a finite 
number of steps provided H~ 1 (O s ) C Q is a finite set which is often the case for discrete systems. 
For other transition systems, there is no guarantee that the algorithm will terminate. In this case, a 
synthesis approach based on approximately bisimilar abstractions can help to compute effectively a 
safety controller with, in addition, an estimation of the distance to maximality. 

3.2. Abstraction-based Controller Synthesis. Let Tj = (Qj, L,5{,0, Hi), i = 1,2, be metric 
transition systems such that T± ~ e T2. Let T\ be the system that we want to control and T2 be an 
approximately bisimilar abstraction of T\. We present an approach to safety controller synthesis for 
specification O s . 

Definition 3.4. Let O'CO and ip > 0. The ip- contraction of O' is the subset of O defined as follows 

C^C) = {o G 0'\ Vo G O, dip, cf)<(p => o G O'} . 
The (p- expansion of O' is the subset of O defined as follows 

EyiO') = {o G 0\ 3o G O', d(o, o') < p} . 

By straightforward applications of the previous definitions, we have: 

Lemma 3.5. Let C C O and e > 0, then C 2£ (0') C C £ (C £ (0')), E e (E £ (0')) C E 2£ {O r ) and 

a c c £ {e £ (o')). 
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We start by synthesizing a safety controller for the abstraction T2 and the specification C e (O s ). This 
controller is denoted 52,c e • We shall not discuss further the synthesis of this controller which can 
be done, if T2 is discrete, using a fixed point algorithm. The second step of our approach allows us 
to design a safety controller for system T\ and specification O s , obtained from the controller S2,c e 
using the following concretization procedure: 

Theorem 3.6. Let T\ ~ e T2, let R C Qi x Q2 denote the e- approximate bisimulation relation between 
T\ and T2. Let S2,c £ be a sa fety controller for T2 and specification C £ {O s ). Let us define S\, the 
controller for T\ given by 

(3.1) Vgi€Qi, S 1 {q l )=S 2 ,cAR(<li))- 

Then, S\ is well-defined and is a safety controller for specification O s . 



Proof. First, let us show that the controller S\ is well-defined. Let q\ E Qi, let I E S\(q\) then, 
from (3.1 ), there exists 52 E Q2 such that (q\, 52) E R and I E c^cvfe)- <S2,c e is well-defined, then I E 



Enab2(<?2), i-e. there exists q' 2 E 62^2, 0- By Definition 2.3, it follows that there exists q[ E 8i(q%,l) 



(such that (q[,q'2) E R), which implies that I E Enabi(qi). Thus, for all q\ E Q\, S\(qi) C Enabi(gi); 
Si is well-defined. Let us now prove that S\ is a safety controller for the specification O s . Let 
qi E Q\ such that S\(qi) / 0, let I E Si(q\), by (3.1) there exists q2 E Q2 such that (91,(72) E R 
and / E ^c^fe)- Since 52,c e is a safety controller for specification C £ (O s ) and 52,c e (<72) 7^ 0, we 



have from Lemma |3.2[ that ^2(92) E C £ (O s ). By Definition 2.3, d(Hi(qi), i^fe)) < £ and therefore 
Hi(qi) E O s . Now, let E <5(c/i, I), by Definition 2.3 there exists q' 2 E i^fe, such that (g' 1; g 2 ) £ R- 



Since 52,c* e is a safety controller for specification C £ (O s ) and I E c^c^fe), we have from Lemma 3.2 
that S 2 ,c e (<?2) ^ 



Finally, (3.1) implies that S2 t c E (q'2) ^= <5i(<zi) an d therefore S±(q[) / 



Lemma |3.2| Si is a safety controller for specification O s 



From 
□ 



If we use the maximal safety controller for T2, it is desirable to have an estimate of the distance 



between the controller given by the concretization equation (3.1) and the maximal safety controller 
for T\. This is given by the following result: 

Theorem 3.7. Let 5^ q and S% Es be the maximal safety controllers for T2 and specifications C e (O s ) 
and E £ (O s ) respectively. Let S\ and Si,e 2£ be the controllers for T\ obtained by the concretization 
equation (3.1) from S^q^ and respectively. Let 5*, S^ q and S^ E2 be the maximal safety 

controllers for T\ and specifications O s , C2 e (O s ) and E2 £ (O s ) respectively. Then, 

S*,c 2e - 5 1 - 5 * - S hE 2s ^ S i,E 2e - 

Proof. The proof relies on the introduction of several auxiliary controllers. The relations between 
these controllers are presented in the following sketch, where arrows correspond to application of the 



concretization equation (3.1): 



-< 



Si 



-< 



Si 



-< 



si 



-< 



-< 



SI 



□ 



Sj d3T 



,E 2e 
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Let us go into the details of the proof. From Theorem 3.6, Si is a safety controller for specification 
O s , then S\ < S\ . Let us prove that 5j* C2 ■< S\ . Since 5* c is a safety controller for specification 

" C 2e (O s 



C 2e (O s ) and from Lemma 3.5 C2 e {O s ) C C £ (C £ (O s )), it is clear that S^ C2 is a safety controller 
for specification C e (C e (O s ))- Now, let us define 52,c e > the controller for T2 such that for q 2 E Q2, 
52,c* e (<?2) = £ic 2 (.R _1 (g2))- The symmetry of approximate bisimulation allows us to reverse the 



<S2,c e (R(qi))- Then, S 2 ^c £ — ^2C E & ves $1 ^ <^i- Finally, we remark that for all gi E Qi 
Sl^{R- l (R{qi)))) which leads to 5^ r< Si. 



Let us show that S* X Si t E 2s - 



Since 5J" 



role of Ti and T2 in Theorem 3.6 This gives that S 2 ^c e is a safety controller for T 2 and specification 
C e (O s ) which yields S 2t c e ^ S 2 c • We now define Si, the controller for T\ such that for E Qi, 
5 x (gi) 
5 x (gi) 

is a safety controller for O s and since from Lemma 3.5 O s C C £ (E e (O s )), it is clear that S± is a 
safety controller for specification C £ (E e (O s )). Now let us define S 2: e £ , the controller for T 2 such 
that for t/2 E Q2> S 2: e e (Q2) = 5;l (-^^fe))- By reversing the role of T\ and T 2 in Theorem 3.6 
we obtain that S 2 ^e e is a safety controller for T 2 and specification E e (O s ). Then, S 2t E £ d> S 2 E ■ 
Then, for all Ql E Qi, 5J(?i) C ^^^-^^(gi))) = <S 2iBe (i?(gi)) C 5^(/J(gi)) = 5 lii?2£ (gi). Hence, 
<5* ^ Sie 2s - Finally, since iS| B is a safety controller for T 2 and specification E e (O s ) and since by 
" E £ (O s ) C C 6 (J5 6 (J5j(O a ))) C C e (^2 £ (Os)), it follows that 5* Bs is a safety controller for 

Sl,E 2e is a safety controller for Ti and 

□ 



3.5 



3.6 



Lemma 

T2 and specification C £ (E 2e (O s )). Then, from Theorem 
specification E 2e (O s ) which yields 5i,£ 2e — 

By computing the controllers 5i and 5i,e 2 e one ^ s able to give a certified upper-bound on the distance 
between the controller Si we will use to control T\ and the maximal safety controller 5j\ Moreover, if 
the safety problem is somehow robust, in the sense that 5j* c and 5* E ^ approach S^ as e approaches 
(i.e. slightly different specifications result in only slightly different maximal controllers); then Si 
and Si t E 2s a lso approach iSj* as e gets smaller and 5j* can be approximated arbitrarily close. 



4. Controller Synthesis for Reachability Specifications 

4.1. Problem Formulation. Let T = (Q, L, 5, O, H) be a transition system, let O s C O be a set of 
outputs associated with safe states, let Ot Q O s be a set of outputs associated with target states. We 
consider the synthesis problem that consists in determining a controller steering the output of the 
system to Ot while keeping the output in O s along the way. In addition, in order to choose among the 
possible controllers, we try to minimize the time to reach the target. Thus, we consider an optimal 
control problem. In this section, we assume for simplicity, that T is non-blocking; it would actually 
be sufficient to assume that all the states of T associated to observations in O s are non-blocking. 

Definition 4.1. Let S be a controller for T such that for all q E Q, S(q) 7^ 0. The entry time 0/T5 
from go E Q for reachability specification (O s , Ot) is the smallest N £N such that for all trajectories 
of the controlled system T|s, of length N and starting from go, (go, lo), (qi,h), • ■ • , (gjv-l; In-i)iQNi 
there exists K E {0, ...,N) such that for all k E {0,...,K}, H(q k ) E O s and H(q K ) E O t . 
The entry time is denoted by J(Ts,O s ,Ot,qo). If such a N E N does not exist, then we define 
J(T s ,O s ,O t ,qo) = +00. 

The condition that 5(g) 7^ 0, for all g E Q, ensures that the controlled system T$ is non-blocking. 
The states from which the system is guaranteed to reach Ot without leaving O s are the states with 
finite entry-time. The following result is quite standard (see e.g. [BerQOj) and is therefore stated 
without proof: 
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Lemma 4.2. The entry time of Tg for reachability specification (O s ,Ot) satisfies: 

• For allqeQ\H- l {O s ), J(T S , O s ,O t ,q) = +00, 
. For allqe H-^Ot), J(T S , O s , O t , q) = 0, 

• For all q G H- l {O s ) \ H- l {O t ), 

(4.1) J(T s ,O s ,O t ,q) = l+ max J(T S , O s , O t , q'). 

q'e6(q,S(q)) 

We can now define the notion of time-optimal controller: 

Definition 4.3. We say that a controller S* for T is time-optimal for reachability specification 
(O s ,Ot) if for all controllers S, for all q £ Q, J{Ts* ,O s ,Ot,q) < J(T$,O s ,Ot,q). The time-optimal 
value function for reachability specification (O s , Ot) is defined as J*(T, O s , Ot, q) = J(T$*, O s , Ot, q). 

Solving the time-optimal control problem consists in synthesizing a time-optimal controller. It is 
well known that a time-optimal controller exists (but may be not unique) and can be computed 
using dynamic programming [BerOO, Tab09]. The dynamic programming algorithm is guaranteed to 
terminate in a finite number of steps provided H~ 1 (O s ) C Q is a finite set which is often the case 
for discrete systems. Here again, for other systems, there is no guarantee that the algorithm will 
terminate and an abstraction-based approach may help to compute a sub-optimal controller with an 
estimation of the distance to optimality. 

4.2. Abstraction-based Controller Synthesis. Let Tj = (Qi, L,5i,0, Hi), i = 1,2, be metric 
transition systems such that T\ ~ £ T2. Let T\ be the system that we want to control and T% be 
an approximately bisimilar abstraction of T\. We present an approach to controller synthesis for 
reachability specifications. We first synthesize a controller 52,c e f° r the abstraction T2 and the 
reachability specification given by the contracted safe set C e (O s ) and target set C e (Ot). If T2 is 
discrete, this can be done using dynamic programming. Then, we design a controller for T\ and 
reachability specification (O s ,Ot) using the following concretization procedure: 

Theorem 4.4. Let T\ ~ e T2, let R C Qi x Q2 denote the e-approximate bisimulation relation between 
T\ andT2- Let 52,c e be a controller for T2, let us define S\, the controller for T\ given bij^ 



(4.2) Vgi G Qi, Si(gi) = S 2 ,c E arg min J(T 2 ,s 2 Cs , C £ (O s ), C £ {O t ),q2, 

V 926-Rwi) 

where q2 £ R{qi) stands for (c/i,^) £ R- Then, S\ is well-defined and for all q\ G Q\: 

(4.3) J(T hSl ,O s ,O t ,qi) < min J(T 2 ,s 2Ce ,C s {O s ),C E (O t ),q 2 ). 

92£-R((ji) 

Proof. Th e fa ct that Si is well-defined can be shown similarly to the proof of Theorem |3.6| Let 



us prove (JL3J), we denote for all qi G Qi, J(q\) = mm q2eR ( qi ) J(T 2i s 2 Ce , C £ (O s ), C £ (O t ), ^2)- If 
^(^l) = 0) then it means that there exists q2 G R(qi) such that J(T2,s 2 Ce > ^ e {O s ), C £ (Ot), q 2 ) = 0. 
This implies that H2{q2) G C £ (Ot). Since (q\, ^2) G R, Definition|2 . 3| gives that d(Hi(q\), ^2(92)) < £• 



Hence Hi(qi) G Ot and J(Ti i s 1 , O s , Ot, qi) = 0. Thus, if J(qi) = 0, (4.3) holds. We now pro- 



ceed by ind uctio n, let us assume that there exists k G N such that for all q\ G Qi such that 



J(<li) < k, (4.3) holds. Let q\ G Q\ such that J{q\) = k + 1, let q2 G Q2 be given by q2 = 
argmin p2eR(gi ) J(T 2j s 2CE ,C £ {O s ),C e (O t ),P2), then, J(T 2: s 2 Ce > C e(O s ),C £ (O t ),q2) = k + 1. This 
implies H 2 (q 2 ) G C £ (O s ) \ C e (O t ). Since ( gi ,q 2 ) G fl, d(H x {q x ) , H 2 (q 2 )) < e and fr a ( gi ) G O s . 



^If there are several states q 2 G R{qi) minimizing J(T 2t s 2 c , C c (O s ), C E (Ot), §2) then we just pick one of them. 
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If Hx( qi ) G O t then J(T 1;Sl ,O s ,O t , qi ) 
assume fli(gi) £ O t . By g2j, 
(91)92) G there exists, by Definition 2.3 
that J(gi) < J(T 2 ^,C £ (O s ),C e (O t ),q' 2 ). 
J(T 2 ,s 2 „,C £ (O s ),C £ (O t ),q' 2 ) < J(T 2 o _ ,C £ (0 



and the induction step is completed. Hence, let us 
^2,C e fe)- Let / G Si(qi), let ^ G £i(gi,Z)> since 
i 92 G ^2(92,0 such that (9i,9 2 ) G i?. It fo llow s 
Since ^2(92) G C £ (O s ) \ C e (O t ), w ^ have by 

C £ (Ot),q 2 ) — l = k. Therefore, J(q[) < k. Then, 



by the induction hypothesis we get that J(Ti )< s 1 , O s , Ot, q[) < J(q[) < Since this holds for all 
I G <Si(<Zi) and all q[ G ^1(91, £), and since H\(q\) G O s \Ot, we have by (4.1 ) that J(Ti t $ 1 , O s , Ot 5 9i) < 
+ 1 which complete s th e induction step. Thus, we h ave p roved by induction that for all q\ G Q\ 
such that J(qi) G N, (|4~3| holds. If J(gi) = +00, then ([43) clearly holds as well. □ 



The previous theorem gives us a way by equation (4.2) to concretize a controller for abstraction T- A 



into a controller for T\. Equation (4.3) provides guarantees on the performance of this controller. 



Particularly, let us remark that the states of Ti i $ 1 from which the control objective is achieved (i.e. 
the states with finite entry-time) are those related through the approximate bisimulation relation R 
to states of T 2 ^ 2 c with finite entry-time. In addition, if S 2) c e is the time-optimal controller for T 2 
and reachability specification (C £ (O s ), C e {Ot)), the following result gives estimates of the distance 
to optimality for the controller S\. 



LetS^c^, <S|s e be time-optimal controllers for T 2 and specification (C e (O s ),C e (Ot)) 



Theorem 4.5 

and (E e (O s ), E £ (Ot)) respectively. Let Si be the controller for T\ obtained from by the con 



cretization equation (4-2). Let 5i,_e 2£ be the controller for T\ obtained from S£ Ee by 



(4.4) y qi G Q u S hEae (qi) = Sl Ee arg min J*(T 2 ,E E (O s ),E £ (O t ),q 2 ) . 

V 92S-R(gi) / 

Then, for all q\ G Q\, 

riTuEtoiO.lEteiP&qx)^ J(T hSlE2£ ,E 2£ (O s ),E 2£ (O t ), gi ) < 

J*(T u O„,O u qx) < 

J(T hSl ,0 Sl O t ,qi) < J*(ri,C 2£ (O s ),C 2e (O t ),9i) 



Proof. The proof essentially follows the same line as the proof of Theorem 3.7 The first and third 
inequalities are direct consequences of the definition of time-optimal value function. Let us prove 
the fourth inequality. Let 5j" C2 be the time-optimal controller for T\ and reachability specification 
(C 2£ (O s ),C 2£ (O t )). From Lemma |3^} we have C 2e (O s ) C C £ (C £ (O s )) and C 2e (O t ) C C £ (C e (O t )). 
Then, for all qi G Q x , J*(T U C 2e {O s ), C 2E (O t ), 9l ) > J(T hS * ,C £ (C £ (O s )),C £ (C £ (O t )), qi ). Now 
let us define the controller for T 2 , S 2) c E , given, for all q 2 G Q 2 , by 

S2,cM = S W farg min J(T liS * ,C £ (C £ (O s )),C £ (C £ (O t )), qi )) . 

By reversing the role of T\ and T 2 in Theorem |4.4[ it follows that for all q 2 G Q 2 , 

J{T 2 . S2Ce ,C £ {O s ),C £ {O t ),q 2 ) < min J(Ti,s* ,C £ (C £ (O s )),C £ (C £ (O t )), qi ) 

< min J*(T 1: C 2£ (O s ),C 2£ (O t ),qi). 

qieR- 1 {q2) 
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Moreover, for all q 2 G Q 2 , J ( T 2,S* Ce , C £ {O s ), C £ {O t ),q 2 ) < J (T 2t s 2)0e , C E (O s ), C e (O t ), q 2 ) which gives 
together with Theorem |4.4[ for all qi G Qi, 



J(T hSl ,O s ,O t , qi ) < win J(T 2j s 2Ce ,C E (O s ),C £ (O t ),q 2 ) 

q2&R(qi) 

< min min J*(T 1} C 2e (O s ), C 2e (O t ),Pi) < J*(T lt C 2e (O s ), C 2e (O t ), Ql ). 

q2eR(qi)pi£R- 1 {q 2 ) 



Let us now prove the second inequality. From Lemma 3.5, O s C C £ (E £ (O s )) and Ot Q C £ (E £ (Ot)). 
Then, for all q x G Q u J*(T 1} O s , O t , ?1 ) > J(T lj5 * , C £ (£ £ (O s )), C £ (£ £ (O t )), ft). Now let us define 
<52,£ e , the controller for T 2 given for all q 2 G Q2 by 

S 2 , Ee (q 2 ) = St farg min J{T l)Sl ,C £ {E £ {O s )),C £ {E £ {O t )), qi )) . 

By reversing the role of T\ and T 2 in Theorem 4.4 it follows that for all q 2 € Q 2 , 

J(T 2>S2Ee ,E e (O s ),E E (O t ),q 2 ) < mm J(T l!S *,C E (E e (O s )),C E {E e (O t )), qi ) 

qieR- 1 (q 2 ) 

< min J*(Ti,O s ,O t ,qi). 

gi6R- 1 (92) 



Using the fact that from Lemma 3.5 E £ (O s ) C C £ {E 2£ {O s )) and E £ {Ot) C C £ (E 2e (Ot)), we can show 
similar to Theorem |4.4| that for all <7i G Qi 

' 2£ q2&R(qi) 

< min J(T 2t s 2Ee ,E e (O s ),E E (O t ),q 2 ) 

q2eR(qi) 

< min min J*(Ti, S , Ot,pi) < J*(Ti, O s , Ot, 91). 

g2ei?(gi)pie-R- 1 (92) 

□ 



By computing the controllers S\ and 5i,E 2e one is able to give a certified upper-bound on the distance 
to optimality of the controller S± we will use to control T\. Moreover, if the reachability problem 
is robust (i.e. the time-optimal value function depends continuously on the specification); then 
J*(Ti, O s , Ot,qi) can be approximated arbitrarily close. We would like to highlight the differences of 
our approach with |MT10| where time-optimal reachability controllers are synthesized using discrete 
abstractions related by alternating simulations. These are weaker assumptions than those considered 
in the present work. In |MT10j . an approach to compute guaranteed upper and lower bounds of the 
value function is given. However, contrarily to our approach there is no clue that these lower and 
upper bounds can approach the true time-optimal value function arbitrarily close. Also, in |MT10j . 
the controllers are refined via the natural concretization procedure that suffers from the drawbacks 
described in Section [2] whereas our approach does not. 

5. Application to Switching Controller Design 

In this section, we present an effective approach to switching controller design based on the syn- 
thesis approaches introduced in this paper in combination with the approximately bisimilar discrete 
abstractions of switched systems developed in [GPTlOj . 

Definition 5.1. A switched system is a triple S = (M n ,P, F), where W 1 is the state space; P = 
{!,..., m} is the finite set of modes; F = {fx, . . . , f m } is a collection of vector fields indexed by P. 
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Given a switched system E = (M n , P, V, F) and a parameter r > 0, we define a transition system 
T T (£) that describes trajectories of duration r of E. This can be seen as a time sampling process. 
This is natural when the switching in E is determined by a time-triggered controller with period 
r. Formally, T r (E) = (Qi, L, 6%, O, Hi) where the set of states is Qi = M. n ; the set of actions is 
L = P; the transition relation is given by x' £ 5i(x,p) if and only if the solution of the differential 
equation x(i) = / p (x(t)) with x(0) = x satisfies x(r) = x'; the set of outputs is O = M n ; the 
observation map Hi is the identity map over W 1 . The set of observations O = W 1 is equipped with 
the metric d(x, x') = \\x — x'\\ where |.| is the usual Euclidean norm. The existence of approximately 
bisimilar discrete abstractions of T r (E) is related to the notion of incremental stability |Ang02|. 
Under this assumption, it is possible to compute approximately bisimilar discrete abstractions of 
arbitrary precision for T r (E) based on a griding of the state-space. Moreover, the approximate 
bisimulation relation can be fully characterized by a Lyapunov function proving incremental stability 
of switched system E (see [GPTlOj for details). 

Controller synthesis for switched systems with safety or reachability specifications can be tackled 
by direct application of fixed-point computation or dynamic programming using guaranteed over- 
approximations [ABD+OOj or convergent approximations [MT00] of reachable sets. In the first case, 
the synthesized controllers are "correct by design" but there is no guarantee that the synthesis 
algorithm will terminate. In the second case, we can only prove that the synthesized controllers are 
"correct in the limit" in the sense that correct controllers can be approximated arbitrarily close. The 
approach described in this paper does not have these problems but only applies to incrementally 
stable systems. For illustration purpose, we apply our approach to a boost DC-DC converter. It is 
a switched system with modes, the two dimensional dynamics associated with both modes are affine 
of the form x(t) = A p x(t) + b for p = 1, 2 (see |GPT10| for numerical values). It can be shown that 
it is incrementally stable and thus approximately bisimilar discrete abstractions can be computed. 

We first consider the problem of regulating the state of the DC-DC converter around a desired 
nominal state. This can be done for instance by synthesizing a controller that keeps the state of the 
switched system in a set centered around the nominal state. This is a safety specification. In the 
following, we consider the specification given by the set O s = [1.1,1.6] x [5.4,5.9]. We use a time 
sampling parameter r = 1 and choose to work with a discrete abstraction that is approximately 
bisimilar to T r (E) with precision e = 0.05. We compute a safety controller for the switched system 



T T (E) by the approach described in Section 3.2 The discrete abstraction has a finite number of states 
inside H^ 1 (C £ (O s )) (actually 169744). The fixed point algorithm for the synthesis of the maximal 
safety controller for the abstraction and specification C e {O s ) terminates in 2 iterations. The resulting 
safety controller Si for the switched system T T (E) and the specification O s is shown on the left part 
of Figure [T] where we have represented a trajectory of the system where the switching is controlled 
using a lazy implementation of the controller S\: when the controller has the choice between mode 
1 and 2, it keeps the current mode active. We can check that the specification is effectively met. We 
also compute the upper-bound of the maximal safety controller S* for switched system T T (E) and 



specification O s , given by Theorem 3.7 The abstraction has 383161 states inside i?^~ 1 (£' e (O s )) and 
the fixed point algorithm for computing the maximal safety controller terminates in 4 iterations. The 
resulting controller Si t E 2s , shown on the right side of Figure [TJ is an upper-bound of the maximal 
safety controller for T T (E) and specification O s . 

We now consider the problem of steering in minimal time the state of the DC-DC converter in the 
desired region of operation while respecting some safety constraints. This is a time-optimal control 
problem. We consider the specification given by the safe set O s = [0.65, 1.65] x [4.95, 5.95] and the 
target set Ot = [1.1,1.6] x [5.4,5.9]. This time, we use a time sampling parameter r = 0.5 and 
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Figure 1. Safety controller Si for the switched system T r (S) and specification O s 
with controlled trajectory (left); Safety controller 5i,e 2e f° r the switched system T T (£) 
and specification E2 £ {O s ) (right); dark gray: mode 1, light gray: mode 2, medium 
gray: both modes are acceptable, white: no action is allowed. The maximal safety 
controller <S* for T T (S) and specification O s satisfies S\ < S* < Si t E 2s - 



choose to work with a discrete abstraction that is approximately bisimilar to T T (E) with precision 
e = 0.1. We compute a suboptimal reachability controller for the switched system T T (£) by the 
approach described in Section |4.2| The discrete abstraction has a finite number of states inside 
H^iC^Os)) (actually 674041). The dynamic programming algorithm for the synthesis of the time- 
optimal controller for the abstraction and reachability specification (C £ (O s ), C £ {Ot)) terminates in 94 
iterations. The resulting suboptimal controller S\ for the switched system T T (£) for the reachability 
specification (O s , Ot) is shown on Figure [2] where we have also represented trajectories of the system 
where the switching is controlled using the synthesized controller. We can check that the specification 
is effectively met. The entry time associated to <Si, J(T T (T,)s 1 , O s , Ot, qi) shown on the left part of 
Figure [3j gives an upper-bound of the time-optimal value function. We also compute the lower-bound 
of the time-optimal value function given by Theorem 4.51 The abstraction has 1520289 states inside 



H2 1 (E £ (O s )) and the fixed point algorithm for computing the maximal safety controller terminates 
in 66 iterations. The resulting controller 5i,_E 2e with entry time J(T T (T,)s 1 E , E2 £ (O s ), E2 £ (Ot), qi) 
provides a lower-bound of the time-optimal value function J*(T r (£), O s , Ot, qi). This lower-bound 
is shown on the right side of Figure [3j 



6. Conclusion 

In this paper, we proposed a methodology, based on the use of approximately bisimilar discrete 
abstractions, for effective computation of controllers for safety and reachability specifications. We 
provided guarantees of performances of the resulting controllers by giving estimates of the distance 
of the synthesized controller to the maximal (i.e the most permissive) safety controller or to the 
time-optimal reachability controller. We showed the effectiveness of our approach by synthesizing 
controllers for a switched system. Let us remark that the techniques presented in the paper are 
independent of the type of abstractions considered as long as these are approximately bisimilar. 
Future work will deal with the development of similar approaches to handle different optimal control 
problems and richer specifications given e.g. in temporal logic. 
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Figure 2. Suboptimal controller Si for the switched system T T (S) and reachability 
specification (O s ,Ot) and trajectories of the controlled switched system. 





Figure 3. Entry-time J(T T (£)s 13 O s , Ot, qi) for the controller Si shown in Figure [2] 
(left); Entry-time (T T (E)s 1 E , E2 E (O s ), E2 £ (Ot), qi) (right); The time-optimal value 
function for the switched system T T (E) and reachability specification (O s , Ot) satisfies 
J(T T (Z) SlE2s ,E2e(O s ),E 2 e(Ot),qi) < J*{T T {Y,),O s ,O t ,qi) < J(T T (E) Sl ,O s ,O t ,qi). 



Acknowledgments. The author would like to thank Gunther Reifiig for his valuable comments on an 
earlier version of this paper. 



References 

[ABD + 00] E. Asarin, O. Bournez, T. Dang, O. Maler, and A. Pnueli. Effective synthesis of switching controllers for 

linear systems. Proc. IEEE, 88(7):1011-1025, 2000. 
[AHLP00] R. Alur, T. Henzinger, G. Lafferriere, and G. J. Pappas. Discrete abstractions of hybrid systems. Proc. of 

the IEEE, 88(2):971-984, 2000. 
[Ang02] D. Angeli. A Lyapunov approach to incremental stability properties. IEEE Trans, on Automatic Control, 

47(3):410-421, March 2002. 

[AVW03] A. Arnold, A. Vincent, and I. Walukiewicz. Games for synthesis of controllers with partial observation. Th. 

Comp. Sc., 28(l):7-34, 2003. 
[BerOO] D. P. Bertsekas. Dynamic Programming and Optimal Control. Athena Scientific, 2000. 



CONTROLLER SYNTHESIS FOR SAFETY AND REACHABILITY VIA APPROXIMATE BISIMULATION 13 



[GirlOa] A. Girard. Synthesis using approximately bisimilar abstractions: state-feedback controllers for safety spec- 
ifications. In Hybrid Systems: Computation and Control, pages 111-120, 2010. 

[GirlOb] Antoine Girard. Synthesis using approximately bisimilar abstractions: Time-optimal control problems. In 
IEEE Con]. Decision and Control, 2010. 

[GP07] A. Girard and G. J. Pappas. Approximation metrics for discrete and continuous systems. IEEE Trans, on 
Automatic Control, 52(5):782-798, 2007. 

[GPT10] A. Girard, G. Pola, and P. Tabuada. Approximately bisimilar symbolic models for incrementally stable 
switched systems. IEEE Trans, on Automatic Control, 55(1):116-126, 2010. 

[HvSOl] L. C. G. J. M. Habets and J. H. van Schuppen. Control of piecewise-linear hybrid systems on simplices and 
rectangles. In Hybrid Systems: Computation and Control, volume 2034 of LNCS, pages 261-274. Springer, 
2001. 

[KB06] M. Kloetzer and C. Belta. A fully automated framework for control of linear systems from ltl specifications. 

In Hybrid Systems: Computation and Control, volume 3927 of LNCS, pages 333-347. Springer, 2006. 
[Mil89] R. Milner. Communication and Concurrency. Prentice Hall, 1989. 

[MR99] T. Moor and J. Raisch. Supervisory control of hybrid systems within a behavioral framework. Systems and 

Control Letters, 38(3): 157-166, 1999. 
[MT00] I. Mitchell and C. Tomlin. Level set methods for computation in hybrid systems. In Hybrid Systems: Com- 
putation and Control, volume 1790 of LNCS, pages 310-323. Springer, 2000. 
[MT10] M. Mazo Jr. and P. Tabuada. Approximate time-optimal control via approximate alternating simulations. 

In American Control Conference, pages 10201-10206, 2010. 
[PGT08] G. Pola, A. Girard, and P. Tabuada. Approximately bisimilar symbolic models for nonlinear control systems. 

Automatica, 44(10):2508-2516, 2008. 
[Rei09] G. Reifiig. Computation of discrete abstractions of arbitrary memory span for nonlinear sampled systems. 

In Hybrid Systems: Computation and Control, volume 5469 of LNCS, pages 306-320. Springer, 2009. 
[R098] J. Raisch and S. O'Young. Discrete approximation and supervisory control of continuous systems. IEEE 

Trans, on Automatic Control, 43(4):569-573, 1998. 
[RW87] P. J. Ramadge and W. M. Wonham. Modular feedback logic for discrete event systems. SIAM J. on Con. 

and Opt, 25(5):1202-1218, 1987. 
[Tab09] P. Tabuada. Verification and Control of Hybrid Systems - A Symbolic Approach. Springer, 2009. 
[TI08] Y. Tazaki and J. I. Imura. Finite abstractions of discrete-time linear systems and its application to optimal 

control. In 17th IFAC World Congress, pages 10201-10206, 2008. 
[TP06] P. Tabuada and G. J. Pappas. Linear time logic control of discrete-time linear systems. IEEE Trans, on 

Automatic Control, 51(12):1862-1877, 2006. 



Laboratoire Jean Kuntzmann, Universite de Grenoble, B.P. 53, 38041 Grenoble, France 



E-mail address: antoine.girard@imag.fr 



